In recent years, distribution systems of data that require realtimeness such as a live image distribution system, video conference system, and the like which exploit an IP (Internet Protocol) network have been proposed.
In general, when many users access the live image distribution system or video conference system, video data suffer some frame drop and audio data suffer sound interruption in some cases. There are a few products that synchronize video and audio data, and in some systems, video and audio data go out of sync after a long time of use. Such problems are posed since video and audio data have different properties.
Video distribution represented by MotionJPEG or the like must transmit a large size of data in a short period of time. For this reason, when video data are distributed to many users while attaching an importance in realtimeness, all data cannot always be transmitted to all users, and the playback side suffers some frame drop upon playback. However, since the human eye can hardly perceive some frame drop of moving image data, such best-effort type transmission method is adopted.
Audio distribution represented by VoIP (Voice over IP) must transmit a smaller size of data than video data at constant intervals over a long period of time. Especially, when audio data are distributed to many users, if all data cannot be transmitted to all playback side clients at constant intervals, sound interruption or generation of undesirable noise such as an echo or the like occurs in some playback side clients.
In the prior art, the playback side client plays back audio data after it buffers the audio data to a given size in consideration of a delay or the like on the network. However, in live distribution that attaches an importance on realtimeness, data for several hundred msec to several sec can only be buffered, and the distribution server side must at least execute transmission processes at constant intervals.
An overview of the operation of the prior art upon distributing video and audio data to many users will be explained below.
A video/audio distribution server generally has a system configuration, as shown in FIG. 1. A distribution server 102 has a plurality of functional blocks.
A plurality of clients 109 transmit video and audio distribution requests to the distribution server 102 via a network 108. In the distribution server 102, a session manager 105 receives transmission requests from the clients via a network unit 107, and creates and saves client information 106. An audio data processor 103 and video data processor 104 respectively generate data to be transmitted, and issue data transmission requests to the network unit 107 via the session manager 105 with reference to the aforementioned client information 106. The network unit 107 delivers data to the clients as destinations in accordance with the transmission requests.
In case of distribution to many users, the audio data processor 103 and video data processor 104 execute processes for transmitting input data to all clients in accordance with the client information 106.
FIG. 2 shows the time relationship between the audio data generation timings and transmission processes when only audio distribution is made to many users in the prior art. In this example, assume that the number N of data to be transmitted at the same time is N=4.
Audio data 200, 201, . . . are generated in turn, and when fourth audio data 203 is generated, the audio data 200 to 203 begin to be transmitted together to all clients. Assume that the transmission completion timing of these data to all the client is the generation timing of audio data 205. In this case, transmission intervals 210 and 211 of the distribution processes to the clients are nearly equal to each other.
FIG. 3 shows all the transmission process timings of the video/audio distribution server when it executes a video distribution process simultaneously with the audio distribution process shown in FIG. 2.
The video and audio distribution processes can parallelly execute transmission processes by means of multi-task processes. However, since only one network unit 107 exists, as shown in FIG. 1, if the transmission process timings overlap, a longer transmission time is required. As can be understood from the example of FIG. 3, a transmission interval 310 is longer than the transmission interval 210 of only the audio distribution process in FIG. 2. On the other hand, since the audio distribution process has a small overlapping part with the video distribution process in a transmission interval 311, it suffers nearly no influence and can complete transmission processes while maintaining the same interval.
As described above, in the prior art, when video and audio distribution processes to many users are done, it is difficult to maintain constant audio transmission intervals. Especially, it becomes more difficult when the processing ability of the transmission apparatus is low.